Information processing apparatus, information processing method, program, and lighting system

ABSTRACT

An information processing apparatus includes a control unit configured to determine a setting of a light emitting unit according to concept data indicating a characteristic of a desired image and a light emission result of the light emitting unit.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, a program, and a lighting system.

BACKGROUND ART

A capture technique of capturing an image in a state of irradiating(lighting) an object with light from a plurality of light sources isknown. Lighting is widely used in production sites for photographs andvideos (for example, movies) as a technique of impressively expressingan object, such as giving a stereoscopic effect to the object. However,to obtain a desired lighting effect, repetition of adjustment for theplurality of light sources and test capture after the adjustment arenecessary, and general lighting systems have a problem of requiring alot of personnel and time. In view of such a situation, there has beenproposed a capture simulation device that can set lighting conditions ona computer and check an image obtained under the set lighting conditionson the computer (see, for example, Patent Document 1 below).

CITATION LIST Patent Document

-   Patent Document 1: Japanese Patent Application Laid-Open No.    2005-301785

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, even if the lighting result can be simulated on the computer asin the technology described in Patent Document 1, there is a problemthat the lighting conditions need to be set through trial and error toobtain a desired lighting effect.

An object of the present disclosure is to provide an informationprocessing apparatus, an information processing method, a program, and alighting system capable of easily performing settings of a lightemitting unit (light source) for obtaining a desired lighting effect.

Solutions to Problems

The present disclosure is, for example,

an information processing apparatus including:

a control unit configured to determine a setting of a light emittingunit according to concept data indicating a characteristic of a desiredimage and a light emission result of the light emitting unit.

The present disclosure is, for example,

an information processing method including: determining a setting of alight emitting unit according to concept data indicating acharacteristic of a desired image and a light emission result of thelight emitting unit.

The present disclosure is, for example,

a program for causing a computer to execute processing of:

determining a setting of a light emitting unit according to concept dataindicating a characteristic of a desired image and a light emissionresult of the light emitting unit.

The present disclosure is, for example,

a lighting system including:

an information processing apparatus; and

a light emitting unit,

the information processing apparatus including

a control unit configured to determine a setting of a light emittingunit according to concept data indicating a characteristic of a desiredimage and a light emission result of the light emitting unit, and

an output unit configured to output the setting to the light emittingunit,

the light emitting unit including

a light emission control unit configured to control light emission basedon the setting output from the information processing apparatus.

Effects of the Invention

According to at least embodiments of the present disclosure, a lightemitting unit for obtaining a desired lighting effect can be easily set.Note that effects described here are not necessarily limited, and any ofeffects described in the present disclosure may be exhibited.Furthermore, content of the present disclosure is not construed in alimited manner by the exemplified effects.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration example of a lightingsystem according to a first embodiment.

FIG. 2 is a block diagram illustrating a configuration example of animaging device according to the first embodiment.

FIG. 3 is a block diagram illustrating a configuration example of atablet computer according to the first embodiment.

FIGS. 4A to 4C are block diagrams illustrating configuration examples oflight sources according to the first embodiment.

FIG. 5 is a diagram for describing calibration processing according tothe first embodiment.

FIG. 6 is a diagram for describing an example of a dimming tableaccording to the first embodiment.

FIG. 7 is a flowchart illustrating an example of a processing flow ofthe calibration processing according to the first embodiment.

FIG. 8 is a diagram for describing an example of a reference image andthe like according to the first embodiment.

FIG. 9 is a flowchart illustrating an example of a processing flow ofconcept data generation processing according to the first embodiment.

FIG. 10 is a flowchart illustrating an example of a processing flow oflighting setting and capture processing according to the firstembodiment.

FIG. 11 is a diagram illustrating a configuration example of a lightingsystem according to a second embodiment.

FIGS. 12A to 12C are block diagrams illustrating a configuration exampleof a light source according to the second embodiment.

FIG. 13 is a flowchart illustrating an example of a processing flow ofcalibration processing according to the second embodiment.

FIG. 14 is a diagram for describing an example of a dimming tableaccording to the second embodiment.

FIG. 15 is a diagram for describing an example of a reference image andthe like according to the second embodiment.

FIG. 16 is a flowchart illustrating an example of a processing flow oflighting setting and capture processing according to the secondembodiment.

FIG. 17 is a diagram for describing modifications.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments and the like of the present disclosure will bedescribed with reference to the drawings. Note that the description willbe given in the following order.

<1. First Embodiment>

≤2. Second Embodiment>

<3. Modification>

Embodiments and the like to be described below are favorable specificexamples of the present disclosure, and content of the presentdisclosure is not limited to these embodiments and the like.

1. First Embodiment

First, a first embodiment according to the present disclosure will bedescribed. The first embodiment is an embodiment for automaticallyperforming a setting of lighting for obtaining a lighting effect desiredby a user when the user captures a still image. The setting of lightingmeans a setting for a light source (light), and in the embodiment, asetting regarding light emission intensity of the light source will bedescribed as an example. Note that the setting for the light source isnot limited to the light emission intensity of the light source. Forexample, the setting may be related to at least one of theabove-described light emission intensity, a color temperature of lightemitted from the light source, softness/hardness of the light, anirradiation range of the light, or an irradiation direction of thelight. The setting may be a specific value (for example, a specificvalue in units of Kelvin (K) in the case of the color temperature) ormay be a value (flag) corresponding to a level of light emissionintensity (for example, strong, medium, or weak of the light emissionintensity) or the irradiation direction of light (for example, slightlyrightward, leftward, or the like with respect to a reference direction).

[Configuration Example of Lighting System]

FIG. 1 illustrates a configuration example of a lighting system(lighting system 10) according to the first embodiment of the presentdisclosure. The lighting system 10 includes an imaging device 1, atablet computer (hereinafter referred to as a tablet computer 2) that isan example of an information processing apparatus, and a plurality oflight sources (light emitting units). The lighting system 10 accordingto the present embodiment includes three light sources (light sources3A, 3B, and 3C) corresponding to an illumination technique calledthree-lamp illumination. For example, the light source 3A is called filllight, the light source 3B is called key light, and the light source 3Cis called backlight. An object 4 is captured using the imaging device 1in a state where the light sources 3A to 3C emit light. In the presentembodiment, the light source 3A and the light source 3B instantaneouslyemit light (flash light emission) in synchronization with capture.Furthermore, the light source 3C does not instantaneously emit light insynchronization with capture although emitting light at the time ofcapture, and is steady light that steadily emits light. The object 4 isschematically illustrated as a person in FIG. 1. However, the object 4is not limited to one person, and may be a plurality of persons, othercreatures such as pets, art works, or the like.

Communication can be performed between the imaging device 1 and thetablet computer 2 and between the tablet computer 2 and the lightsources 3A to 3C. For example, wireless communication based on WiFi(registered trademark) is performed between the devices. Wirelesscommunication based on a communication standard other than WiFi(registered trademark) may be performed, communication using infraredrays may be performed, or wired communication may be performed. Forexample, these devices communicate with each other after performingknown pairing processing for enabling these devices to recognize eachother and become able to communicate with each other.

[Imaging Device]

(Configuration Example of Imaging Device)

Next, configuration examples of the devices constituting the lightingsystem 10 will be described. First, a configuration example of theimaging device 1 will be described. FIG. 2 illustrates a configurationexample of the imaging device 1. The imaging device 1 includes, forexample, an optical imaging system 101, a dedicated phase differenceauto focus (AF) sensor 102, an imaging element 103, a preprocessingcircuit 104, a camera processing circuit 105, an image memory 106, and acamera control unit 107, a graphic interface (I/F) 108, a display 109,an input unit 110, a reader/writer (R/W) 111, a storage unit 112, and acamera communication unit 113.

The optical imaging system 101 includes a capture lens 101A (including afocus lens, a zoom lens, and the like) for condensing light from anobject on the imaging element 103, a lens drive mechanism 101B thatperforms focus adjustment by moving the focus lens, and a shuttermechanism, an iris mechanism, and the like. These lens and mechanismsare driven on the basis of a control signal from the camera control unit107. The lens drive mechanism 101B realizes an AF operation by movingthe capture lens 101A along an optical axis direction in accordance withthe control signal supplied from the camera control unit 107. An opticalimage of the object obtained through the optical imaging system 101 isformed on the imaging element 103 as an imaging device.

The dedicated phase difference AF sensor 102 is, for example, a phasedifference detection-type auto focus dedicated sensor. Object lightcondensed by the capture lens 101A is reflected by a semi-transmissivemirror or the like to enter the dedicated phase difference AF sensor102. A focus detection signal detected by the dedicated phase differenceAF sensor 102 is supplied to the camera control unit 107.

The imaging element 103 is a charge coupled device (CCD), acomplementary metal oxide semiconductor (CMOS), or the like. The imagingelement 103 photoelectrically converts the object light incident via thecapture lens 101A into a charge amount, and generates an image. Theimaging element 103 has a normal imaging pixel and a phase differencedetection pixel. An image plane phase difference AF sensor 103A is anautofocus sensor including a plurality of phase difference detectionpixels. The focus detection signal detected by the image plane phasedifference AF sensor 103A is supplied to the camera control unit 107.

The preprocessing circuit 104 performs sample hold for an imaging signaloutput from the imaging element 103 to maintain a favorable signal/noise(S/N) ratio by correlated double sampling (CDS) processing. Moreover,the preprocessing circuit 104 controls a gain by auto gain control (AGC)processing and performs analog/digital (A/D) conversion, and outputs adigital image signal.

The camera processing circuit 105 applies signal processing to the imagesignal from the preprocessing circuit 104, such as white balanceadjustment processing, color correction processing, gamma correctionprocessing, Y/C conversion processing, and auto exposure (AE)processing.

The image memory 106 is a volatile memory, for example, a buffer memoryconfigured by a dynamic random access memory (DRAM), and temporarilystores image data to which the predetermined processing has been appliedby the preprocessing circuit 104 and the camera processing circuit 105.

The camera control unit 107 is configured by, for example, a centralprocessing unit (CPU) and includes a read only memory (ROM) 107A and arandom access memory (RAM) 107B. The ROM 107A stores a program that isread and executed by the CPU and the like. The RAM 107B is used as awork memory for the CPU. The CPU controls the entire imaging device 1 byexecuting various types of processing in accordance with the programstored in the ROM 107A and issuing commands. For example, the CPUperforms control regarding the above-described AF. Although details willbe described below, the camera control unit 107 controls each unit ofthe imaging device 1 so that capture is performed in a state where eachof the plurality of light sources 3A to 3C emits light on the basis ofthe corresponding setting.

The graphic I/F 108 generates an image signal to be displayed on thedisplay 109 from the image signal supplied from the camera control unit107 and supplies the image signal to the display 109 to display animage. The display 109 displays a through image being captured, an imagerecorded in the storage unit 112, or the like.

The display 109 is configured by a liquid crystal display (LCD), anorganic electroluminescence (EL) display, or the like.

The input unit 110 includes, for example, a power button for switchingon/off of power, a release button for giving an instruction on start ofrecording of a captured image, an operator for zoom adjustment, a touchscreen integrally configured with the display 109, and the like. When aninput is made to the input unit 110, a control signal corresponding tothe input is generated and output to the camera control unit 107. Then,the camera control unit 107 performs arithmetic processing and controlcorresponding to the control signal.

The R/W 111 is an interface to which the storage unit 112 that recordsimage data generated by imaging is connected. The R/W 111 writes datasupplied from the camera control unit 107 to the storage unit 112, andoutputs the data read from the storage unit 112 to the camera controlunit 107. The storage unit 112 is, for example, a hard disk, a memorystick (registered trademark of Sony Corporation), an SD memory card, auniversal serial bus (USB) memory, or the like. An image is stored in acompressed state on the basis of a standard of joint photographicexperts group (JPEG) or the like. Furthermore, exchangeable image fileformat (EXIF) data including information regarding the stored image andadditional information such as capture date and time is also stored inassociation with the image. The storage unit 112 may be built in theimaging device 1, may be attachable/detachable to/from the imagingdevice 1, or may be both.

The camera communication unit 113 is a configuration for the imagingdevice 1 to communicate with other devices (for example, the tabletcomputer 2), and has functions necessary for communication such as anantenna and a modulation/demodulation function. For example, the imagedata of the image captured by the imaging device 1 is transmitted fromthe imaging device 1 to the tablet computer 2 by the cameracommunication unit 113.

(Basic Operation Example of Imaging Device)

Here, a basic operation in the above-described imaging device 1 will bedescribed. Before capture of an image, signals received andphotoelectrically converted by the imaging element 103 are sequentiallysupplied to the preprocessing circuit 104. The preprocessing circuit 104applies CDS processing, AGC processing, and the like to the inputsignal, and further converts the input signal into an image signal.

The camera processing circuit 105 performs image quality correctionprocessing for the image signal supplied from the preprocessing circuit104, and supplies the image signal to the graphic I/F 108 via the cameracontrol unit 107 as a through image signal. Thereby, the through imageis displayed on the display 109. The user views the through imagedisplayed on the display 109 and can adjust an angle of view.

In this state, when the shutter button of the input unit 110 is pressed,the camera control unit 107 outputs the control signal to the opticalimaging system 101 to operate shutters constituting the optical imagingsystem 101. Thereby, the imaging element 103 outputs an image signal forone frame.

The camera processing circuit 105 applies image quality correctionprocessing to the image signal for one frame supplied from the imagingelement 103 via the preprocessing circuit 104, and supplies theprocessed image signal to the camera control unit 107. The cameracontrol unit 107 compresses and encodes the input image signal, andsupplies the generated coded data to the R/W 111. Thereby, a data fileof a captured still image is stored in the storage unit 112 via the R/W111. Note that, in capturing a moving image, the above-describedprocessing is performed in real time in response to a moving imagecapture instruction. A still image can be captured at the time ofcapturing a moving image by pressing the shutter button at the time ofcapturing the moving image.

Meanwhile, in a case of reproducing an image file stored in the storageunit 112, the camera control unit 107 reads a selected still image filefrom the storage unit 112 via the R/W 111 in response to an operationinput from the input unit 110. Decompression decoding processing isapplied to the read image file. Then, the decoded image signal issupplied to the graphic I/F 108 via the camera control unit 107.Thereby, a still image or a moving image stored in the storage unit 112is displayed on the display 109.

[Configuration Example of Tablet Computer]

FIG. 3 illustrates a configuration example of a tablet computer 2. Thetablet computer 2 has a portable size, and includes, for example, acontrol unit 201, a communication unit (hereinafter referred to as atablet communication unit) 202 included in the tablet computer, adisplay unit 203, an input unit 204, an R/W 205, and a storage unit 206.

The control unit 201 includes a CPU and the like, and includes a ROM anda RAM (not illustrated). The ROM stores a program that is read andexecuted by the CPU and the like. The RAM is used as a work memory forthe CPU. Furthermore, the control unit 201 includes a dimming tablegeneration unit 201A and a dimming information generation unit 201B asfunctions. The control unit 201 determines the settings of the lightsources 3A to 3C according to concept data indicating an imagecharacteristic desired by the user and light emission results of thelight sources 3A to 3C.

The tablet communication unit 202 as an example of an output unit is aconfiguration for the tablet computer 2 to communicate with otherdevices (for example, the imaging device 1 and the light sources 3A to3C), and has functions necessary for communication such as an antennaand a modulation/demodulation function. For example, the image data ofthe image transmitted from the imaging device 1 is received by thetablet communication unit 202, and the received image data is suppliedto the control unit 201. Furthermore, the setting for each of the lightsources 3A to 3C is transmitted from the tablet communication unit 202to the corresponding light source. Note that the tablet communicationunit 202 according to the present embodiment also has a function to beconnected to a network such as the Internet.

The display unit 203 is configured by a liquid crystal display, anorganic EL display, or the like. The display unit 203 displays imagesand videos acquired via the Internet, television broadcasts, gamescreens, menu screens, and the like. Furthermore, the display unit 203displays a reference image that is referred to when the user accepts aninput for generating concept data to be described below. Note that thedisplay unit 203 according to the present embodiment is configured as atouch screen, and is configured to accept various inputs. That is, thedisplay unit 203 also functions as the input unit 204.

The input unit 204 is, for example, a physical button such as a pushablebutton or a slide button, a touch screen, or the like, and is acollective term for configurations that accept user operation inputs.When an input is made to the input unit 204, a control signalcorresponding to the input is generated and output to the control unit201. Then, the control unit 201 performs arithmetic processing andcontrol corresponding to the control signal. The input unit 204 is alsoused for accepting an input for generating the concept data to bedescribed below.

The R/W 205 writes data to the storage unit 206 and reads data from thestorage unit 206. The storage unit 206 is, for example, a hard disk, amemory stick (registered trademark of Sony Corporation), an SD memorycard, a USB memory, or the like. The storage unit 206 may be built inthe tablet computer 2, may be attachable/detachable to/from the tabletcomputer 2, or may be both.

Note that the tablet computer 2 may have a configuration other than theabove-described configuration. For example, the tablet computer 2 mayinclude a speaker device or the like. An operation example of the tabletcomputer 2 will be described below.

[Configuration Example of Light Source]

FIG. 4 illustrates configuration examples of the light sources, and FIG.4A illustrates a configuration example of the light source 3A, FIG. 4Billustrates a configuration example of the light source 3B, and FIG. 4Cillustrates a configuration example of the light source 3C.

As illustrated in FIG. 4A, the light source 3A includes, for example, alight emitting unit (light emitting body) 301A, a dimming control unit302A, and a light source communication unit 303A. The light emittingunit 301A is, for example, a flash light source such as a strobe, andemits light in synchronization with capture by the imaging device 1. Thedimming control unit 302A controls the light emission intensity, lightemission timing, and the like of the light emitting unit 301A on thebasis of dimming information received by the light source communicationunit 303A. The light source communication unit 303A is used for thelight source 3A to communicate with other devices (for example, theimaging device 1 and the tablet computer 2).

As illustrated in FIG. 4B, the light source 3B includes, for example, alight emitting unit (light emitting body) 301B, a dimming control unit302B, and a light source communication unit 303B. The light emittingunit 301B is, for example, a flash light source such as a strobe, andemits light in synchronization with capture by the imaging device 1. Thedimming control unit 302B controls the light emission intensity, lightemission timing, and the like of the light emitting unit 301B on thebasis of dimming information received by the light source communicationunit 303B. The light source communication unit 303B is used for thelight source 3B to communicate with other devices (for example, theimaging device 1 and the tablet computer 2).

As illustrated in FIG. 4C, the light source 3C includes, for example, alight emitting unit (light emitting body) 301C, a dimming control unit302C, and a light source communication unit 303C. The light emittingunit 301C is, for example, a fluorescent lamp or a light emitting diode(LED). The dimming control unit 302C controls the light emissionintensity of the light emitting unit 301C on the basis of dimminginformation received by the light source communication unit 303C. Thelight source communication unit 303C is used for the light source 3C tocommunicate with other devices (for example, the imaging device 1 andthe tablet computer 2). Note that it goes without saying that theconfigurations of the light sources 3A to 3C may be different.

As an example, each of the light sources 3A to 3C is configured to emitlight at three levels of light emission intensity (weak, medium, andstrong). Of course, the light sources 3A to 3C may emit light at twolevels of light emission intensity, or at multi levels such as four ormore levels of light emission intensity. Furthermore, for example, thelight emission intensity “weak” of the light source 3A and the lightemission intensity “weak” of the light source 3B may be the same lightemission intensity or different levels of light emission intensity.

[Processing Performed in First Embodiment]

Next, processing performed in the first embodiment will be described,including an operation of each device. In the present embodiment, forconvenience of description, the processing performed in the firstembodiment is divided into three types of processing: “calibrationprocessing”, “concept data generation processing”, and “lighting settingand capture processing”, and each type of the processing will bedescribed in turn. Note that the three types of processing do not needto be performed in the order described below, and the order of theprocessing may be changed within a range where no technicalcontradiction occurs, the processing may be performed in parallel, oreach type of the processing may be continuously performed. For example,the calibration processing may be performed after the concept datageneration processing, or the calibration processing and the conceptdata generation processing may be performed in parallel.

(Calibration Processing)

The calibration processing is processing of measuring in advance aninfluence of light by light emission on the image of the object in acase where the light emission result of each light source, specifically,each light source emits light in a different pattern on a one-by-onebasis, and obtaining a table that is a measurement result (hereinafterthe table is referred to as a dimming table as appropriate).

The calibration processing will be schematically described withreference to FIG. 5. First, the light source 3A emits light in differentlight emission patterns. In the present example, the light source 3Aemits light at different levels of light emission intensity. However,the different light emission patterns may be different colortemperatures or the like. This also similarly applies to the other lightsources. Note that the light sources 3B and 3C remain off withoutemitting light.

The light source 3A emits light with the light emission intensity “weak”in synchronization with capture, and capture is performed. Next, thelight source 3A emits light with the light emission intensity “medium”in synchronization with capture, and capture is performed. Next, thelight source 3A emits light with the light emission intensity “strong”in synchronization with capture, and capture is performed. Imagesobtained by the captures are temporarily stored in, for example, theimage memory 106 of the imaging device 1.

Next, the light source 3B emits light at different levels of lightemission intensity. Note that the light sources 3A and 3C remain offwithout emitting light. The light source 3B emits light with the lightemission intensity “weak” in synchronization with capture, and captureis performed. Next, the light source 3B emits light with the lightemission intensity “medium” in synchronization with capture, and captureis performed. Next, the light source 3B emits light with the lightemission intensity “strong” in synchronization with capture, and captureis performed. Images obtained by the captures are temporarily stored in,for example, the image memory 106 of the imaging device 1.

Next, the light source 3C emits light at different levels of lightemission intensity. Note that the light sources 3A and 3B remain offwithout emitting light. Capture is performed in a state where the lightsource 3C emits light with the light emission intensity “weak”. Next,capture is performed in a state where the light source 3C emits lightwith the light emission intensity “medium”. Capture is performed in astate where the light source 3C emits light with the light emissionintensity “strong”. Images obtained by the captures are temporarilystored in, for example, the image memory 106 of the imaging device 1.

Image data of nine images obtained by the captures is transmitted fromthe imaging device 1 to the tablet computer 2. That is, the image datais transmitted from the camera communication unit 113, and the imagedata is received by the tablet communication unit 202. The receivedimage data is supplied from the tablet communication unit 202 to thecontrol unit 201. The dimming table generation unit 201A of the controlunit 201 analyzes and acquires image information for each of the nineimages.

The image information is spatial sensor information corresponding to thelight emission pattern of each light source. In the present embodiment,a luminance level for each pixel constituting the image is used. Notethat, to reduce the amount of information, a luminance level in a unitof a block having an arbitrary size instead of for each pixel, or datacompressed by another appropriate method may be used as the imageinformation. Furthermore, the processing of analyzing and acquiring theimage information may be performed by the imaging device 1, and only theimage information may be transmitted from the imaging device 1 to thetablet computer 2.

The dimming table generation unit 201A, which has acquired the imageinformation, generates a dimming table corresponding to the lightemission result of each light source. FIG. 6 illustrates an example of adimming table (dimming table TA1) according to the first embodiment. Thedimming table TA1 includes, for example, the type of light source(information such as an identifier that can identify the light source),the light emission intensity, and the image information (in the presentembodiment, the luminance level of each pixel of the image captured withthe light emission intensity) that are associated with one another.

Note that the dimming table TA1 generated by the dimming tablegeneration unit 201A may be written and stored in the storage unit 206via the R/W 205. Thereby, for example, when the same or similar objectis captured in the same studio, the calibration processing can beomitted by using the dimming table TA1 stored in the storage unit 206.The dimming table TA1 may be stored in a memory (for example, a memoryon the cloud) that is not the storage unit 206, or may be transmittedand distributed to other devices by communication.

Furthermore, for example, in a case where change in the imageinformation with respect to change in the light emission intensity isknown, image information corresponding to light emission intensitydifferent from predetermined light emission intensity may be generatedusing image information corresponding to the predetermined lightemission intensity in order to simplify the calibration processing. Forexample, regarding the light source 3A, the dimming table generationunit 201A may predict and generate the image information correspondingto the light emission intensity “medium” and to the light emissionintensity “strong” on the basis of the image information in the case ofemitting light with the light emission intensity “weak”. Thereby, thenumber of captures can be reduced, and the calibration processing can besimplified.

Note that the object (object to be captured) in the calibrationprocessing is favorably the same as but may be different from the objectin the “lighting setting and capture processing”. For example, theobject in the calibration processing may be an object similar to theobject in the “lighting setting and capture processing” (a person with asimilar face size or a similar skin color in a case where the object isa person).

FIG. 7 is a flowchart illustrating an example of a processing flow ofthe calibration processing. In step ST11, an instruction to emit lightwith the light emission intensity “weak” is given to a predeterminedlight source (light source 3A, for example). This instruction is madeusing the input unit 204 of the tablet computer 2, for example. Notethat, since the light source 3A emits light in synchronization withcapture, synchronization information such as timing to synchronize lightemission and capture is transmitted from the tablet computer 2 to thelight source 3A and the imaging device 1. Then, the processing proceedsto steps ST12 and ST13.

In steps ST12 and ST13, capture is performed at the timing indicated bythe synchronization information, and the light source 3A emits lightwith the light emission intensity “weak” in synchronization with thecapture.

Specifically, the camera control unit 107 of the imaging device 1controls each unit of the imaging device 1 so that capture is performedat the timing indicated by the synchronization information. Furthermore,the dimming control unit 302A of the light source 3A causes the lightemitting unit 301A to emit light with the light emission intensity“weak” at the timing indicated by the synchronization information. Then,the processing proceeds to step ST14.

In step ST14, an image obtained by capture in step ST13 is transmittedfrom the imaging device 1 to the tablet computer 2. Then, the dimmingtable generation unit 201A of the tablet computer 2 generates oneelement of the dimming table TA1 by analyzing the image information ofthe image in the case where the light source 3A emits light with thelight emission intensity “weak”. Then, the processing proceeds to stepST15.

In step ST15, an instruction to emit light with the light emissionintensity “medium” is given to the light source 3A. This instruction isalso given using the input unit 204 of the tablet computer 2, forexample. Note that, as described above, since the light source 3A emitslight in synchronization with capture, the synchronization informationsuch as timing to synchronize light emission and capture is transmittedfrom the tablet computer 2 to the light source 3A and the imaging device1. Then, the processing proceeds to steps ST16 and ST17.

In steps ST16 and ST17, capture is performed at the timing indicated bythe synchronization information, and the light source 3A emits lightwith the light emission intensity “medium” in synchronization with thecapture. Specifically, the camera control unit 107 of the imaging device1 controls each unit of the imaging device 1 so that capture isperformed at the timing indicated by the synchronization information.Furthermore, the dimming control unit 302A of the light source 3A causesthe light emitting unit 301A to emit light with the light emissionintensity “medium” at the timing indicated by the synchronizationinformation. Then, the processing proceeds to step ST18.

In step ST18, an image obtained by capture in step ST16 is transmittedfrom the imaging device 1 to the tablet computer 2. Then, the dimmingtable generation unit 201A of the tablet computer 2 generates oneelement of the dimming table TA1 by analyzing the image information ofthe image in the case where the light source 3A emits light with thelight emission intensity “medium”. Then, the processing proceeds to stepST19.

In step ST19, an instruction to emit light with the light emissionintensity “strong” is given to the light source 3A. This instruction isalso given using the input unit 204 of the tablet computer 2, forexample. Note that, as described above, since the light source 3A emitslight in synchronization with capture, the synchronization informationsuch as timing to synchronize light emission and capture is transmittedfrom the tablet computer 2 to the light source 3A and the imaging device1. Then, the processing proceeds to steps ST20 and ST21.

In steps ST20 and ST21, capture is performed at the timing indicated bythe synchronization information, and the light source 3A emits lightwith the light emission intensity “strong” in synchronization with thecapture. Specifically, the camera control unit 107 of the imaging device1 controls each unit of the imaging device 1 so that capture isperformed at the timing indicated by the synchronization information.Furthermore, the dimming control unit 302A of the light source 3A causesthe light emitting unit 301A to emit light with the light emissionintensity “strong” at the timing indicated by the synchronizationinformation. Then, the processing proceeds to step ST22.

In step ST22, an image obtained by capture in step ST21 is transmittedfrom the imaging device 1 to the tablet computer 2. Then, the dimmingtable generation unit 201A of the tablet computer 2 generates oneelement of the dimming table TA1 by analyzing the image information ofthe image in the case where the light source 3A emits light with thelight emission intensity “strong”. Then, the processing proceeds to stepST23.

In step ST23, whether or not all pieces of the processing using thelight sources have been completed is determined. In the present example,processing using the light source 3B and the light source 3C has notbeen completed, the processing returns to step ST11. Then, similarly tothe above-described processing, the light source 3B is caused to emitlight at different levels of light emission intensity, and one elementof the dimming table TA1 is generated on the basis of the imageinformation of the image obtained at that time. Similar processing isperformed for the light source 3C. Note that, since the light source 3Cis not flash light emission, capture may be performed by operating theinput unit 110 of the imaging device 1 in a state of causing the lightsource 3C to emit light with predetermined light emission intensity.That is, the synchronization information does not need to be transmittedto the imaging device 1 and the light source 3C.

In step ST23, in a case where all pieces of the processing using thelight sources have been completed, the dimming table TA1 illustrated inFIG. 6 is obtained on the basis of the elements of the dimming tableobtained in the respective pieces of processing in steps ST14, ST18, andST22. Then, the processing ends.

(Concept Data Generation Processing)

Next, the concept data generation processing will be described. Theconcept data is data indicating an image characteristic desired (by theuser), and more specifically, data indicating brightness for each areaof the object.

The concept data is specified using the input unit 204 of the tabletcomputer 2, for example. The display unit 203 displays the referenceimage that is referred to when an input to the input unit 204 isaccepted. For example, as illustrated in FIG. 8, a reference image IM1is displayed on the display unit 203. The user performs an operationbased on his/her concept for the input unit 204 while referring to thereference image IM1.

As the reference image IM1, for example, one of the nine images obtainedby the above-described calibration processing can be used. Thiseliminates the need to capture the reference image IM1 itself.Furthermore, the reference image IM1 may be an image obtained bycapturing the object captured by the “lighting setting and captureprocessing”. The capture at this time may be performed in a state whereonly ambient light (indoor room light not related to capture) is emittedin a state where the light sources 3A to 3C do not emit light. Moreover,the reference image IM1 may be a template image selected according to apredetermined result of known person recognition or face recognitionwith respect to the object captured in the “lighting setting and captureprocessing”. In this case, the predetermined processing is performed bythe control unit 201, for example. Furthermore, the template image maybe an image obtained by reading the image stored in the storage unit 206by the R/W 205, or may be an image acquired from the Internet or thelike.

FIG. 9 is a flowchart illustrating an example of a processing flow ofthe concept data generation processing. A specific example of theconcept data generation processing will be described with reference toFIGS. 8 and 9. The concept data generation processing is started when,for example, a predetermined button of the input unit 204 is pressed ortouched. In step ST31, the reference image IM1 is displayed on thedisplay unit 203. Then, the processing proceeds to step ST32.

In step ST32, the user performs an input based on the concept whilereferring to the reference image IM1. For example, as illustrated inFIG. 8, two areas AR1 and AR2 are set for the object. For example,brightness is specified such that the brightness of the area AR1 becomesappropriate exposure and the brightness of the area AR2 becomes 2/3(about 66%) of appropriate exposure. The setting of the areas and thespecification of the brightness are performed using the input unit 204.In the present embodiment, the configuration (for example, thebrightness) corresponding to the input is reflected in the referenceimage IM1 every time input is performed by the input unit 204. Forexample, as illustrated in FIG. 8, the brightness of the area AR2becomes slightly dark. Note that settings of the brightness of abackground and the like may be performed although not illustrated. Whenthe user finishes the specification of the brightness or the like, theuser presses or touches a completion button or the like to terminate theinput based on the concept. Then, the processing proceeds to step ST33.

In step ST33, concept data is generated on the basis of the input madein step ST32. The concept data is generated by the control unit 201, forexample. As described above, the concept data is data indicating that,for example, a right half of the face is properly exposed and a lefthalf of the face is exposed by 2/3 of the appropriate exposure. Then,the processing ends.

Note that the concept data may be stored in the storage unit 206. Then,when capturing an object with a similar face size, skin color, or thelike, the previous concept data may be read from the storage unit 206and used. Furthermore, the concept data may be distributed bycommunication or the like via the tablet communication unit 202. Forexample, the concept data of a well-known photographer has a proprietaryvalue in itself. Therefore, by charging for the distribution of theconcept data, a new business model using the concept data as a keybecomes able to be provided.

(Lighting Setting and Capture Processing)

Next, the lighting setting and capture processing will be described.FIG. 10 is a flowchart illustrating an example of a processing flow ofthe lighting setting and capture processing. After the process isstarted, in step ST41, the dimming information generation unit 201Bcalculates and determines the light emission intensity of each lightsource according to the concept data and the dimming table TA1. Theinfluence of light in a case where each light source emits light withcertain light emission intensity on the image of the object is alreadyknown according to the dimming table TA1. Moreover, for example, in acase where light from the light source 3A and light from the lightsource 3B are emitted on a predetermined location, the brightness of thelocation becomes a sum of the brightness. For this reason, the dimminginformation generation unit 201B can determine how much light emissionintensity is required for each light source to emit light in order torealize the brightness for each area specified in the concept data byreference to the dimming table TA1.

In the case of the concept data illustrated in FIG. 8, for example, thedimming information of the light source 3A is generated such that thelight emission intensity of the light source 3A becomes “weak”, thedimming information of the light source 3B is generated such that thelight emission intensity of the light source 3B becomes “strong”, andthe dimming information of the light source 3C is generated such thatthe light emission intensity of the light source 3C becomes “medium”.Note that, in a case where a composition that completely matches thecomposition based on the concept data cannot be realized, the dimminginformation is generated to realize a composition as close as possibleto the composition based on the concept data. As an optimization methodof obtaining an optimum lighting setting (dimming information) for theconcept data, a quasi-Newton method, a downhill simplex method, a dualsimplex method, a sequential linear quadratic programming, or the likecan be applied. Then, the processing proceeds to step ST42.

In step ST42, the tablet computer 2 outputs the dimming informationindicating the light emission intensity of each light source to thecorresponding light source. For example, the tablet computer 2 transmitsthe dimming information to the corresponding light source, using thetablet communication unit 202. Then, the processing proceeds to stepST43.

In step ST43, the object 4 is captured in a state where each lightsource emits light on the basis of the dimming information. For example,the dimming information corresponding to the light source 3A is receivedby the light source communication unit 303A. The dimming control unit302A causes the light emitting unit 301A to emit light with the lightemission intensity “weak” on the basis of the received dimminginformation. Furthermore, the dimming information corresponding to thelight source 3B is received by the light source communication unit 303B.The dimming control unit 302B causes the light emitting unit 301B toemit light with the light emission intensity “strong” on the basis ofthe received dimming information. Furthermore, the dimming informationcorresponding to the light source 3C is received by the light sourcecommunication unit 303C. The dimming control unit 302C causes the lightemitting unit 301C to emit light with the light emission intensity“medium” on the basis of the received dimming information. Then, captureby the imaging device 1 is performed in the state where each lightsource emits light. An image obtained by the capture is displayed on thedisplay 109 of the imaging device 1. The image obtained by the capturemay be transmitted from the imaging device 1 to the tablet computer 2and displayed on the display unit 203. Then, the processing proceeds tostep ST44.

In step ST44, adjustment (fine adjustment) by the user is performed asnecessary. As the adjustment, the user himself or herself may directlyadjust the light emission intensity of some or all of the light sources.Furthermore, the concept data itself may be finely adjusted. Then,dimming information may be generated again on the basis of the conceptdata after fine adjustment, and capture may be performed again in thestate where each light source emits light with the light emissionintensity corresponding to the dimming information. Moreover, capturewith changed light emission intensity of each light source may beperformed a plurality of times.

Note that, in the above-described processing, capture and light emissionneed to be synchronized because the light source 3A and the light source3B are flash light emission. The synchronization information forsynchronization may be included in the dimming information or a lightemission instruction may be given to the light source 3A and the lightsource 3B when the release button of the imaging device 1 is pressed ina state where the light source 3C continues to emit light. Furthermore,in the above-described processing, the dimming information for eachlight source has been generated. However, the dimming information may beinformation in which the light emission intensity of all the lightsources is written, and each light source may select its own lightemission intensity from the dimming information.

Example of Effect Obtained in First Embodiment

According to the above-described first embodiment, the settings of thelight sources for obtaining a lighting effect desired by the user can beautomatically performed. Furthermore, since a person who adjusts thelight source is unnecessary, capture by a small number of people (oneperson in the minimum) is possible even if the lighting system becomeslarge.

Regardless of whether the setting is manually performed or performed bya simulation using a computer, the setting of the light source to obtaina desired lighting effect takes time and effort. Since a generallighting system uses a plurality of light sources, the settings takesmore time and effort. However, according to the present embodiment, theuser can reduce the time and effort required for the adjustment of thelight sources and can perform efficient capture because the settings ofthe light sources can be performed by simply setting the concept datacorresponding to the result of lighting capture, in other words, adesired image configuration (for example, the brightness for each areaof the object). Furthermore, since the number of trials to adjust thelight sources and cause the light sources to emit light can be reduced,the load on the light sources can be reduced and the durability of thelight sources can be improved.

Furthermore, since capture is actually performed under lightingoptimized for the concept, a high-quality image (still image in thepresent embodiment) that cannot be realized by image editing softwarecan be obtained.

2. Second Embodiment

Next, a second embodiment according to the present disclosure will bedescribed. Note that the matters described in the first embodiment canbe applied to the second embodiment unless otherwise specified, andsimilarly, the same reference numerals are given to configurations ofthe same nature, and redundant description is appropriately omitted.

[Problems to Consider when Capturing Moving Image Under Lighting]

The second embodiment is an embodiment for automatically performing asetting of lighting for obtaining a lighting effect desired by a userwhen the user captures a moving image. Basic processing is similar tothe processing in the first embodiment, and the setting for each lightsource is automatically performed on the basis of concept data set bythe user.

By the way, in a case of capturing a moving image under lighting, aviewpoint specific to the moving image needs to be considered. FIG. 11illustrates a lighting system (lighting system 10A) according to thesecond embodiment in which a moving image is captured under lighting.The lighting system 10A includes an imaging device 1 and a tabletcomputer 2 described in the first embodiment. Moreover, the lightingsystem 10A includes, for example, a light source 5A (fill light), alight source 5B (key light), and a light source 5C (backlight) as lightsources. Note that, since a moving image is captured in the presentembodiment, the light sources 5A to 5C emit steady light rather thanflash light emission (instantaneous light)

In capturing a moving image, there are generally an object that moves(moving object) such as a person and an object that does not move suchas a background. In the lighting system 10A illustrated in FIG. 11, aperson 6 is illustrated as an example of the moving object and a wall 7is illustrated as the background. The person 6 is mainly irradiated withlight from the light source 5A and the light source 5B, and the wall 7is mainly irradiated with light from the light source 5C.

The present embodiment assumes a case of capturing a state where theperson 6 moves in a depth direction, more specifically, from the rear(depth side) toward the front (front side) with respect to the imagingdevice 1. It is assumed that the user desires a composition in which thebrightness of the person 6 (brightness of reflected light) does notchange with the movement of the person 6. In such a case, a methodcalled a wiping method is generally used in which a light amount isadjusted to be constant while covering the light source with a diffuser.However, the wiping method requires a highly skilled technique.Therefore, in a case of arranging personnel capable of performing thewipe method in each light source, personnel capable of performing thewipe method need to be secured as a premise. Furthermore, the cost incapturing the moving image increases due to rewards and the like for thepersonnel. As described above, in the case of capturing a moving imageunder lighting, the problem of the increase in personnel and theincrease in cost associated with the increase in personnel becomes morepronounced than the case of capturing a still image under lighting.Details of the second embodiment will be described on the basis of sucha viewpoint.

[Configurations of Devices]

Since configurations of the imaging device 1 and the tablet computer 2have been respectively described with reference to FIGS. 2 and 3,redundant description is omitted. Note that, although details will bedescribed below, a control unit 201 controls determined settingsregarding the light sources 5A to 5C to maintain structure data in acase where an object changes when capturing a moving image.

FIGS. 12A to 12C illustrate configuration examples of the light sources5A to 5C. The light source 5A includes a light emitting unit 501A, adimming control unit 502A, and a light source communication unit 503A,similarly to the light source 3A. The light source 5B includes a lightemitting unit 501B, a dimming control unit 502B, and a light sourcecommunication unit 503B, similarly to the light source 3B. The lightsource 5C includes a light emitting unit 501C, a dimming control unit502C, and a light source communication unit 503C, similarly to the lightsource 3C. Note that, since the function of each unit is the same as thefunction of each unit in the light source 3A, redundant description isomitted. The light sources 5A to 5C are configured to emit light atthree levels of light emission intensity, for example, “weak”, “medium”,and “strong”.

[Processing Performed in Second Embodiment]

Next, processing performed in the second embodiment will be described,including an operation of each device. In the second embodiment, forconvenience of description, processing is divided into three types ofprocessing: “calibration processing”, “concept data generationprocessing”, and “lighting setting and capture processing”, and theneach type of the processing will be described in turn. The three typesof processing do not need to be performed in the order described below,and the order of the processing may be changed within a range where notechnical contradiction occurs, the processing may be performed inparallel, or each type of the processing may be continuously performed.

(Calibration Processing)

First, the calibration processing according to the second embodimentwill be described. FIG. 13 is a flowchart illustrating an example of aprocessing flow of the calibration processing according to the secondembodiment. Note that processing similar to the processing described inthe flowchart in FIG. 7 is denoted by the same reference numerals, andredundant description is omitted as appropriate.

After the processing is started, in step ST51, the person 6 as themoving object moves to a location AA. The location AA is, for example, aposition where the person 6 starts moving. Then, in steps ST11 to ST22,for example, the light source 5A sequentially emits light with the lightemission intensity “low”, “medium”, and “strong”, and capture in eachlight emission state is performed. This capture may be a still image ora moving image. In each capture, image information corresponding to eachlight emission intensity is analyzed, and one element of a dimming tableis generated by a dimming table generation unit 201A. When capture iscompleted, the processing proceeds to step ST23.

In step ST23, whether or not all the processing (capture) using eachlight source at a predetermined location (location AA in the presentexample) has been completed is determined. In the present example,processing using the light sources 5B and 5C has not been completed, theprocessing returns to step ST11. Then, in steps ST11 to ST22, forexample, the light source 5B sequentially emits light with the lightemission intensity “low”, “medium”, and “strong”, and capture in eachlight emission state is performed. Furthermore, for example, the lightsource 5C sequentially emits light with the light emission intensity“low”, “medium”, and “strong”, and capture in each light emission stateis performed. In each capture, image information corresponding to eachlight emission intensity is analyzed, and one element of a dimming tableis generated by a dimming table generation unit 201A. When capture iscompleted, the processing proceeds to step ST23.

Since the capture using the light sources 5A, 5B, and 5C in the statewhere the person 6 is in the location AA has been completed,determination in the processing in step ST23 is affirmative, and theprocess proceeds to step ST52. In step ST52, whether or not capture atall of locations has been completed is determined. In the presentexample, determination in the processing in step ST52 is negative, andthe processing proceeds to step ST53.

In step ST53, the person 6 as a moving object moves to a location BB.The location BB is, for example, a position where the person 6 hasfinished moving, that is, a final stop position. Then, the processingreturns to step ST11. Then, in steps ST11 to ST22, for example, thelight source 5A sequentially emits light with the light emissionintensity “low”, “medium”, and “strong”, and capture in each lightemission state is performed. In each capture, image informationcorresponding to each light emission intensity is analyzed, and oneelement of a dimming table is generated by a dimming table generationunit 201A. Similar processing is performed for the other light sources5B and 5C. Then, the processing proceeds to step ST23.

In step ST23, whether or not all the processing (capture) using eachlight source at a predetermined location (location BB in the presentexample) has been completed is determined. In a case where capture usingthe light sources 5A, 5B, and 5C is completed, the processing proceedsto step ST52. As described above, in step ST52, whether or not captureat all of locations has been completed is determined. In the presentexample, capture locations are the two locations of the location AA thatis a movement start position of the person 6 and the location BB that isa movement stop position. Therefore, the determination in step ST52 isaffirmative, and the process ends.

FIG. 14 illustrates an example of a dimming table (dimming table TA2)obtained by the calibration processing according to the secondembodiment. The dimming table TA2 is a table in which the type of lightsource, the light emission intensity, position information of the person6, and a luminance level of each pixel of an image captured at that timeare associated with one another. Note that the position information ofthe person 6 can be expressed by various types of information. Forexample, known person extraction processing is performed on the basis ofan image in capture, an area occupied by the person 6 (area where theperson 6 is present) at the location AA is obtained, and the area may beused as the position information. Furthermore, distance information ofthe person 6 at each location may be used as the position information onthe basis of a depth map obtained by a distance-measuring sensor or byimage processing. Furthermore, the person 6 moves while holding a beaconand information transmitted from the beacon at each location may be usedas the position information. Furthermore, combined information of theabove pieces of information may be used as the position information.

Note that the dimming table TA2 has a larger information amount than thedimming table TA1 described in the first embodiment. Simply stated, theinformation is increased by the number of capture locations, and in theabove-described example, capture is performed at the location AA andlocation BB, so the information amount in the dimming table TA2 is abouttwice the information amount in the dimming table TA1. Therefore, forexample, the luminance level of each pixel of the dimming table TA2 maybe compressed using a known compression technology. Furthermore, onlythe luminance levels of the pixels existing in an area set by conceptdata to be described below may be described in the dimming table TA2.

(Concept Data Generation Processing)

Next, the concept data generation processing according to the secondembodiment will be described. The processing flow of the concept datageneration processing according to the second embodiment is basicallythe same as the concept data generation processing according to thefirst embodiment. That is, a setting of the concept data using an inputunit 204 is performed by the user in a state where a reference image isdisplayed on a display unit 203 of a tablet computer 2.

FIG. 15 is a diagram for describing an example of the concept dataaccording to the second embodiment. A reference image IM2 is displayedon the display unit 203. The reference image IM2 is, for example, animage (still image) when the user is located at the location AA. Thisimage may be an image obtained by the calibration processing or may be aseparately captured image.

The user inputs the concept data using the input unit 204. For example,an area AR3 including the person 6 and a background area AR4 arespecified. Then, for example, the concept data for maintaining thebrightness of the person 6 and the background (the intensity ofreflected light from the objects) is set by the user. Here, thebrightness of the person 6 and the background at the capture startposition (the state in which the moving object is at the location AA,which will be referred to as an initial state as appropriate in thefollowing description) is also set as the concept data.

Note that the setting of the concept data is an example and is notlimited to this example. For example, in the above-described example, arectangular area has been illustrated. However, a person may beextracted and a setting for the person itself (for example, maintainingthe brightness of the person) may be made. Furthermore, as in theabove-described example, in the case of maintaining the brightness ofthe moving object and the background, concept data of “maintaining thebrightness of all of objects (including the background)” may be simplyset without setting areas. The object to be set in the concept data isset not only by the user but may also be automatically recognized byperson extraction or face recognition.

(Lighting Setting and Capture Processing)

Next, the lighting setting and capture processing according to thesecond embodiment will be described. FIG. 16 is a flowchart illustratingan example of a processing flow of the lighting setting and captureprocessing. After the processing is started, in step ST61, a dimminginformation generation unit 201B generates dimming informationindicating the light emission intensity of each light source on thebasis of the concept data and the dimming table TA2. That is, thedimming information generation unit 201B generates the dimminginformation for each light source while referring to the dimming tableTA2 such that the brightness of the person 6 and the background matchesor approaches the brightness in the initial state indicated by theconcept data. As described in the first embodiment, as an optimizationmethod of obtaining an optimum lighting setting (dimming information)for the concept data, a quasi-Newton method, a downhill simplex method,a dual simplex method, a sequential linear quadratic programming, or thelike can be applied. Then, the processing proceeds to step ST62.

In step ST62, the tablet computer 2 outputs the dimming informationindicating the light emission intensity of each light source to thecorresponding light source. For example, the tablet computer 2 transmitsthe dimming information to the corresponding light source, using thetablet communication unit 202. The dimming information corresponding tothe light source 5A is received by the light source communication unit503A. The dimming control unit 502A controls the light emissionintensity of the light emitting unit 501A on the basis of the receiveddimming information. The dimming information corresponding to the lightsource 5B is received by the light source communication unit 503B. Thedimming control unit 502B controls the light emission intensity of thelight emitting unit 501B on the basis of the received dimminginformation. The dimming information corresponding to the light source5C is received by the light source communication unit 503C. The dimmingcontrol unit 502C controls the light emission intensity of the lightemitting unit 501C on the basis of the received dimming information.Then, the processing proceeds to step ST63.

In step ST63, capture of a moving image by the imaging device 1 isstarted in the state where each light source emits light on the basis ofthe dimming information, and an image for one frame is obtained as aresult of the capture. The obtained image is transmitted from theimaging device 1 to the tablet computer 2. After the transmitted imageis received by the tablet communication unit 202, the image is suppliedto the control unit 201. Then, the processing proceeds to step ST64.

In step ST64, the control unit 201 calculates a difference between theimage of one frame obtained in step ST64 and the concept data. Anexample of the difference calculation method will be described. Thecontrol unit 201 extracts a portion corresponding to the area AR3including the person 6 from the image of one frame. Then, the controlunit 201 compares an average value of the brightness (luminance) of eachpixel in the area AR3 with an average value of the brightness of eachpixel in the area AR3 in the initial state, and calculates thedifference. Similar calculation is performed for the area AR4. Then, theprocessing proceeds to step ST65.

In step ST65, whether or not the difference regarding at least one ofthe areas AR3 or AR4 calculated in step ST64 is equal to or larger thana predetermined threshold value is determined. For example, thepredetermined threshold value is appropriately set to a value thatallows change in brightness to be visually recognized. In a case wherethe difference is not equal to or larger than the predeterminedthreshold value, the processing proceeds to step ST66.

In step ST66, whether or not capture of the moving image has beencompleted is determined. When capture of the moving image has beencompleted, the processing proceeds to step ST67, and adjustment (fineadjustment) for the captured moving image is performed as necessary. Inthe present example, since capture of only one frame has been completed,the determination in step ST66 is negative, and the processing returnsto step ST63. Then, the processing of steps ST63 and ST64 is performedfor one frame to be captured next.

The present embodiment assumes the case in which the person 6 moves fromthe depth toward the front, as described above. As the person 6approaches the front side, the distance between the light source 5A andthe light source 5B and the person 6 decreases, in other words, theperson 6 becomes brighter. Therefore, the processing in step ST65becomes affirmed at certain timing. In this case, the processing returnsto step ST61.

In step ST61, the dimming information generation unit 201B recalculatesthe dimming information. That is, the dimming information generationunit 201B recalculates the dimming information while referring to thedimming table TA2 such that the brightness of the person 6 and thebackground matches the brightness indicated by the concept data at thecurrent position of the person 6. Conceptually, the light emissionintensity of the light source 5A and the light source 5B is reduced.However, the brightness of the background decreases, and the conceptdata cannot be realized. Accordingly, the light emission intensity ofthe light source 5C is increased. Thereby, in a case where change occursdue to the movement of the person 6, the setting regarding each lightsource can be controlled to maintain the concept data. Morespecifically, the brightness of the person 6 and the background can bemaintained according to the concept data. Then, the processing proceedsto step ST62, and the dimming information indicating the light emissionintensity of each light source is output (transmitted) from the tabletcomputer 2 to each corresponding light source. Then, capture of a movingimage is performed in a state where each light source emits light on thebasis of the recalculated dimming information. The above-describedprocessing is repeated during the capture of the moving image.

Note that, in the dimming table TA2 illustrated in FIG. 14, only imageinformation of when the person 6 is present at the two locations AA andBB is described. Therefore, an influence of light emission with lightemission intensity by a certain light source on the object at a locationbetween the location AA and the location BB (hereinafter, the locationis referred to as a location CC as appropriate. Note that the locationCC can be an arbitrary position) can be interpolated using theinformation of the location AA and the location BB. For example, in acase where the light source 5A emits light with the light emissionintensity “weak”, the luminance level of a pixel in a range occupied bythe person 6 at the location CC may be predicted using the luminancelevel of a pixel in a case where the light source 5A emits light withthe light emission intensity “weak” in a state where the person 6 ispresent at the location AA, and the luminance level of a pixel in a casewhere the light source 5A emits light with the light emission intensity“weak” in a state where the person 6 is present at the location BB.Furthermore, the luminance level of a pixel in a case where a certainlight source emits light with certain light emission intensity may bepredicted in a state where the person 6 is present at the location CC,using change in spatial position of the person 6 (change in depthinformation).

Note that a plurality of the locations CC is set, and the luminancelevel of each pixel in a case where a certain light source emits lightwith certain light emission intensity may be acquired in advance by theabove-described calibration processing at the location CC. Thereby, theload of the processing of recalculating the dimming information can bereduced. The setting of the plurality of locations CC between thelocation AA and the location BB may be set by the user or may beautomatically set as appropriate. For example, in a case where changeequal to or larger than a threshold value is detected in the brightnessof the person 6 in a case where the light source 3A emits light with thelight emission intensity “weak”, position information of the position,the light source, the light emission intensity of the light source, andthe luminance level of each pixel may be described in the dimming tableTA2 in association with one another.

Note that the degree of change in brightness associated with movementbetween frames is slight. Therefore, the processing in steps ST63 toST65 may be performed in arbitrary units (for example, in units of 10frames) rather than in units of one frame.

Example of Effect Obtained in Second Embodiment

By performing the above-described processing, the settings of the lightsources are automatically performed to maintain the concept data even ina case where a moving object is present in an object, and a differenceoccurs between an image obtained according to movement of the movingobject and the concept data. Therefore, according to the secondembodiment, the settings of the light sources for obtaining a lightingeffect desired by the user can be automatically performed even in thecase of capturing a moving image. Furthermore, a moving image thatmatches the concept data can be obtained regardless of the speed atwhich the person 6 moves. Moreover, similar effects to the effectsexemplified in the above-described first embodiment can be obtained.

3. Modification

The plurality of embodiments of the present disclosure has beenspecifically described. However, content of the present disclosure isnot limited to the above-described embodiments, and variousmodifications based on the technical idea of the present disclosure canbe made. Hereinafter, modifications will be described. Note that themodifications to be described below can be applied to the first andsecond embodiments unless otherwise specified or within a range where notechnical contradiction occurs.

In the above-described embodiments, a plurality of concept data may beset, and a plurality of pieces of dimming information corresponding tothe respective concept data may be generated. Then, capture may beperformed a plurality of times in a state where the light sources emitlight on the basis of the respective plurality of pieces of dimminginformation.

An image based on the concept data and an image obtained by actualcapture may be different by a predetermined amount or more due to aperformance limit of a light source or the like. In such a case, awarning or an error may be given in notification. As an example, pixelvalues of pixels constituting the image based on the concept data andpixel values of pixels constituting the image obtained by actual captureare compared, and in a case where there is a certain number of pixelshaving a difference that is equal to or larger than a threshold value,the images are determined to be different by a predetermined amount ormore. Notification of a warning or an error is performed by sound,display, vibration, or the like.

In the above-described embodiments, settings such as a shutter speed,sensitivity, and an aperture of the imaging device 1 may be changed.Such settings for the imaging device 1 may be performed according touser's preference such as widening the aperture to blur the background,or may be performed to compensate for the performance limit of a lightsource.

In the above-described embodiments, the light sources may be built inthe imaging device 1 or attachable/detachable to/from the imaging device1, or may be lights that are small enough to be held in hand by theuser. Furthermore, capture may be performed indoors or outdoors, andpart of the light sources may be ambient light (natural light) such assunlight. Furthermore, the shapes, sizes, and arrangement positions ofthe light sources are merely schematically illustrated, and are notlimited to the examples illustrated in the above-described embodiments.Furthermore, the number of light sources is not limited to three.Although several tens of light sources may be used in a large-scaleproduction site, the present technology can be applied to such alarge-scale lighting system. Furthermore, part of the light sources maybe ambient light, and another one or a plurality of the light sourcesmay be lights.

In the above-described embodiments, the processing of obtaining thedimming information may be performed by another device (for example, acloud computer) instead of the tablet computer 2. Furthermore, theprocessing of obtaining the dimming information may be performed by theimaging device 1. Furthermore, the imaging device 1 or other device mayhave a part or all of the functions of the tablet computer 2.Furthermore, in a case where the tablet computer 2 has an imagingfunction, the tablet computer 2 may capture images instead of theimaging device 1. In this case, the control unit 201 of the tabletcomputer 2 may have the function of the camera control unit 107 of theimaging device 1. As described above, which device has the configurationthat realizes the functions in the above-described embodiments can bechanged as appropriate.

The configuration for performing communication (for example, the tabletcommunication unit 202) may be a communication adapter or the like thatis attachable/detachable to/from the tablet computer 2.

The location where the input using the tablet computer 2 is performedand the location where the capture is performed may not be in the samespace but may be different locations. For example, a person responsiblefor determining the concept inputs the concept data using the tabletcomputer 2 at home or the like, and transmits the concept data to astudio where the capture is performed. A device installed in the studiomay perform capture after generating the dimming information for eachlight source on the basis of the dimming table and the concept data andcontrolling the light emission intensity of the light source and thelike. This eliminates the need for the person responsible fordetermining the concept to go to the location where the capture isperformed.

In the above-described second embodiment, an example of maintaining thebrightness of the person and the background has been described as anexample of the concept data. However, the concept data is not limited tothe example. For example, the concept data may be concept data in whichchange in brightness is specified. Specifically, concept data in which aperson becoming brighter or darker with movement of the person isspecified may be the concept data. Then, as described in theabove-described second embodiment, the setting of each light source isappropriately changed to cancel a difference from the concept datacaused by the movement of the person, and the concept data is realized.Moreover, the concept data may include, for example, time informationsuch as how much a person becomes bright after a predetermined time.

In the second embodiment, in a case where a recording rate of theimaging device 1 is 60 frames per second (fps) and an imaging rate ishigh speed such as 120 fps or 240 fps, the calibration processing andthe processing of recalculating the dimming information may be performedusing an image not used for recording.

In the above-described second embodiment, the movement of the person 6has been described using the front-rear direction as an example.However, a left-right direction, a diagonal direction, a combineddirection of the aforementioned directions, or the like may be used.

The present disclosure can also be realized by a device, a method, aprogram, a system, and the like. For example, a program for performingthe functions described in the embodiments is made downloadable, and adevice not having the control function described in the embodimentsdownloads and installs the program, thereby becoming able to perform thecontrol described in the embodiments. The present disclosure can also berealized by a server that distributes such a program.

In the above-described embodiments, the number of light emissionpatterns of each light source in the calibration processing is set toone type, and light emission intensity obtained by a through the lens(TTL) automatic dimming function of the imaging device may be used asthe light emission intensity of the light emission pattern of each lightsource. At that time, it is favorable to perform capture under onlyambient light that is not emitted by all the light sources.

In the above-described embodiments, the user may be notified in a casewhere an image obtained by capture in the calibration processingincludes an area that is over-exposed or under-exposed, exceeding anupper limit or a lower limit of the luminance value. The image obtainedby capture may be displayed at the time of notification and the areathat is over-exposed or under-exposed may be displayed in an emphasizedmanner.

In the above-described embodiments, in a case where the colortemperature of each light source in the calibration processing isdifferent, the color temperature of each light may be multiplied by acoefficient to suppress an influence by a different in color temperatureand used. Furthermore, in a case where the color temperature isdifferent depending on the light emission amount even in the same lightsource, the color temperature may be multiplied by a coefficientaccording to the light emission amount and used.

In the above-described embodiments, an example of creating the dimmingtable from the image obtained by capture in the calibration processinghas been described. However, the embodiments are not limited to theexample. For example, the image information obtained by capture may beused as it is without being converted into the dimming table. The imageinformation used at that time is favorably a raw image for securing thenumber of colors represented by a bit depth of a simulation image andgradation in order to avoid an influence of gamma.

In the above-described embodiments, an example of using the optimizationmethod as described above for obtaining optimum dimming information inthe lighting setting and capture processing has been described. However,in a case of applying the image information obtained by capture as it isto the lighting setting and capture processing, the lighting setting andcapture processing may be performed by a different method. Specifically,a simulation image of results of lighting capture is generated bycomparative bright combination of images obtained by capture, and theuser confirms the generated image. As a result of the confirmation, in acase where the generated image is a desired simulation image, thedimming information for obtaining a capture result similar to thegenerated simulation image is transmitted to each light source inresponse to a user operation. On the other hand, in a case where thegenerated image is not the desired simulation image as a result of theconfirmation, adjustment is performed to obtain a simulation imagedesired by the user by changing a coefficient for the luminance value ofeach image to be used for comparative bright combination in response tothe user operation. The dimming information for obtaining a captureresult similar to the simulation image after user adjustment istransmitted to each light source in response to the user operation aftercompletion of the user adjustment.

By using the described-above comparative bright combination, an ambientlight component and a light emission component from each light sourcecan be separately calculated. Corresponding pixel values are comparedbetween a captured image under only the ambient light and a lightemitted image, the light emission component can be obtained by acquiringa larger pixel value. Since the relationship between the light emissionintensity of a light source and obtained reflected light is representedby a linear function with a slope of 1 in a log-log graph, the lightemission component can be calculated. This greatly reduces thecalculation amount in the simulation. Furthermore, as the method ofcomparative bright combination, color comparative bright combinationthat performs comparison with a total value of RGB may be used.

Furthermore, the simulation image may be displayed on the display unit203 for the user to confirm the simulation image by the comparativebright combination described above, as illustrated in FIG. 17, forexample. At that time, the image captured under only ambient light maybe displayed as the reference image IM3 in the initial state, or asimulation image by preset predetermined comparative bright combinationmay be displayed. At that time, a comparative bright combination imageselection unit 601 for selecting image information to be used forcomparative bright combination and a coefficient setting unit 602 forsetting a coefficient for the luminance value of each piece of imageinformation may be simultaneously displayed with the IM3. Thereby, theuser can operate the comparative bright combination image selection unit601 and the coefficient setting unit 602 while referring to thereference image IM3, thereby adjusting a desired simulation image basedon the user's own concept. Furthermore, the display unit 203 may displaya light source information display unit 603 for displaying informationof each connected light source, a recalibration instruction unit 604 forgiving an instruction on recalibration, a main capture instruction unit605 for transmitting the dimming information to each light source andproceeding to main capture, and a simulation image output unit 606 foroutputting a simulation image being displayed as a completed image.

Here, the comparative bright combination may be performed between theimage information captured under only ambient light and light emissionpattern image information of each light source, or may be performedbetween the image information captured under only ambient light andimage information of a difference in luminance value between the lightemission pattern image information of each light source and the imageinformation captured under only ambient light. Thereby, an influence ofa difference of the ambient light depending on the time of capturing theimage information to be used for comparative bright combination can bereduced.

In the above-described embodiments, when change in a light exposurestate of an object is detected by each light source, the imaging device,or the information processing apparatus, part or all of the calibrationprocessing, the concept data generation processing, and the lightingsetting and capture processing may be performed again. Alternatively,part or all of the processes of the calibration processing, the conceptdata generation processing, and the lighting setting and captureprocessing may be performed again by a user operation. Here, the lightexposure state of an object can be considered to change by change inincrease/decrease in the number of light sources, movement of theposition of the object or the light source, change in reflectance of theobject itself, change in a setting value of the imaging device or thelight source, attachment/detachment of an accessory such as a diffuser,or change in weather or ambient light. Each light source, the imagingdevice, or the information processing apparatus detects the change inthe light exposure state of the object by detection of a user operation,various sensors, change in a captured image, or the like. In the case ofdetecting the change in the light exposure state of the object by thedetection of a user operation, various sensors, change in a capturedimage, or the like, part or all of various types of processing may beautomatically performed again, or the change in the light exposure stateof the object may be given in notification to the user.

The present disclosure can also employ the following configurations.

(1)

An information processing apparatus including:

a control unit configured to determine a setting of a light emittingunit according to concept data indicating a characteristic of a desiredimage and a light emission result of the light emitting unit.

(2)

The information processing apparatus according to (1), in which

the concept data is data indicating brightness specified for each areaof the object.

(3)

The information processing apparatus according to (2), in which

the concept data includes data indicating first brightness specified fora first area of the object, and data indicating second brightnessspecified for a second area of the object.

(4)

The information processing apparatus according to (3), in which

the data indicating the first brightness is data corresponding toappropriate exposure, and the data indicating the second brightness is arelative value with respect to the appropriate exposure.

(5)

The information processing apparatus according to any one of (1) to (4),in which

the control unit generates a dimming table corresponding to the lightemission result of the light emitting unit on the basis of imageinformation of an image obtained in a case where the light emitting unithas emitted light in a predetermined light emission pattern.

(6)

The information processing apparatus according to (5), in which

the light emission pattern is light emission intensity of the lightemitting unit.

(7)

The information processing apparatus according to (6), in which

the control unit uses image information corresponding to predeterminedlight emission intensity to generate image information corresponding tolight emission intensity different from the predetermined light emissionintensity.

(8)

The information processing apparatus according to (5), in which

the light emission pattern is a color temperature of the light emittingunit.

(9)

The information processing apparatus according to any one of (5) to (8),in which

the image information is spatial sensor information corresponding to thelight emission pattern of the light emitting unit.

(10)

The information processing apparatus according to (9), in which

the image information is a luminance level in a pixel unit or a unit ofa block having an arbitrary size.

(11)

The information processing apparatus according to any one of (1) to(10), further including:

an input unit configured to accept an input for generating the conceptdata.

(12)

The information processing apparatus according to (11), furtherincluding:

a display unit configured to display a reference image to be referred towhen the input is accepted.

(13)

The information processing apparatus according to (12), in which

a configuration corresponding to the input is reflected in the referenceimage in response to the input.

(14)

The information processing apparatus according to (12) or (13), in which

the reference image is any one of an image obtained when the lightemission result of the light emitting unit is acquired, an imagecaptured in a state where the light emitting unit is not emitting light,and a template image selected according to a result of predeterminedprocessing.

(15)

The information processing apparatus according to any one of (1) to(14), in which

the setting of the light emitting unit is a setting regarding at leastone of light emission intensity of the light emitting unit, a colortemperature of light emitted from the light emitting unit,softness/hardness of the light, an irradiation range of the light, or anirradiation direction of the light.

(16)

The information processing apparatus according to any one of (1) to(15), further including:

an output unit configured to output the setting of the light emittingunit.

(17)

An information processing method including:

determining a setting of a light emitting unit according to concept dataindicating a characteristic of a desired image and a light emissionresult of the light emitting unit.

(18)

A program for causing a computer to execute processing of:

determining a setting of a light emitting unit according to concept dataindicating a characteristic of a desired image and a light emissionresult of the light emitting unit.

(19)

A lighting system including:

an information processing apparatus; and

a light emitting unit,

the information processing apparatus including

a control unit configured to determine a setting of a light emittingunit according to concept data indicating a characteristic of a desiredimage and a light emission result of the light emitting unit, and

an output unit configured to output the setting to the light emittingunit,

the light emitting unit including

a light emission control unit configured to control light emission basedon the setting output from the information processing apparatus.

(20)

The lighting system according to (19), in which a part of a plurality ofthe light emitting units emits light in synchronization with thecapture.

REFERENCE SIGNS LIST

-   1 Imaging device-   2 Tablet computer-   3A to 3C, 5A to 5C Light source-   10 Lighting system-   107 Camera control unit-   201 Control unit-   201A Dimming table generation unit-   201B Dimming information generation unit-   202 Tablet communication unit-   203 Display unit-   204 Input unit-   303A to 303C, 503 to 503C Light source communication unit

1. An information processing apparatus comprising: a control unitconfigured to determine a setting of a light emitting unit according toconcept data indicating a characteristic of a desired image and a lightemission result of the light emitting unit.
 2. The informationprocessing apparatus according to claim 1, wherein the concept data isdata indicating brightness specified for each area of an object.
 3. Theinformation processing apparatus according to claim 2, wherein theconcept data includes data indicating first brightness specified for afirst area of the object, and data indicating second brightnessspecified for a second area of the object.
 4. The information processingapparatus according to claim 3, wherein the data indicating the firstbrightness is data corresponding to appropriate exposure, and the dataindicating the second brightness is a relative value with respect to theappropriate exposure.
 5. The information processing apparatus accordingto claim 1, wherein the control unit generates a dimming tablecorresponding to the light emission result of the light emitting unit ona basis of image information of an image obtained in a case where thelight emitting unit has emitted light in a predetermined light emissionpattern.
 6. The information processing apparatus according to claim 5,wherein the light emission pattern is light emission intensity of thelight emitting unit.
 7. The information processing apparatus accordingto claim 6, wherein the control unit uses image informationcorresponding to predetermined light emission intensity to generateimage information corresponding to light emission intensity differentfrom the predetermined light emission intensity.
 8. The informationprocessing apparatus according to claim 5, wherein the light emissionpattern is a color temperature of the light emitting unit.
 9. Theinformation processing apparatus according to claim 5, wherein the imageinformation is spatial sensor information corresponding to the lightemission pattern of the light emitting unit.
 10. The informationprocessing apparatus according to claim 9, wherein the image informationis a luminance level in a pixel unit or a unit of a block having anarbitrary size.
 11. The information processing apparatus according toclaim 1, further comprising: an input unit configured to accept an inputfor generating the concept data.
 12. The information processingapparatus according to claim 11, further comprising: a display unitconfigured to display a reference image to be referred to when the inputis accepted.
 13. The information processing apparatus according to claim12, wherein a configuration corresponding to the input is reflected inthe reference image in response to the input.
 14. The informationprocessing apparatus according to claim 12, wherein the reference imageis any one of an image obtained when the light emission result of thelight emitting unit is acquired, an image captured in a state where thelight emitting unit is not emitting light, and a template image selectedaccording to a result of predetermined processing.
 15. The informationprocessing apparatus according to claim 1, wherein the setting of thelight emitting unit is a setting regarding at least one of lightemission intensity of the light emitting unit, a color temperature oflight emitted from the light emitting unit, softness/hardness of thelight, an irradiation range of the light, or an irradiation direction ofthe light.
 16. The information processing apparatus according to claim1, further comprising: an output unit configured to output the settingof the light emitting unit.
 17. An information processing methodcomprising: determining a setting of a light emitting unit according toconcept data indicating a characteristic of a desired image and a lightemission result of the light emitting unit.
 18. A program for causing acomputer to execute processing of: determining a setting of a lightemitting unit according to concept data indicating a characteristic of adesired image and a light emission result of the light emitting unit.19. A lighting system comprising: an information processing apparatus;and a light emitting unit, the information processing apparatusincluding a control unit configured to determine a setting of a lightemitting unit according to concept data indicating a characteristic of adesired image and a light emission result of the light emitting unit,and an output unit configured to output the setting to the lightemitting unit, the light emitting unit including a light emissioncontrol unit configured to control light emission based on the settingoutput from the information processing apparatus.
 20. The lightingsystem according to claim 19, wherein a part of a plurality of the lightemitting units emits light in synchronization with the capture.